A number of artificial intelligence based reasoning systems exist for analyzing text-based or semantic network-based representations of data to extract desired information from the data such as determining an answer to a question pertaining to a given subject-matter associated with the data. These reasoning systems typically include an application executing on data processing hardware. Sometimes problems posed to such systems have an image component as well. For example, some questions from medical licensing exams include annotated anatomy diagrams or radiology images. Unfortunately, no straightforward way exists to convert these semantic-based (e.g., language-based) networks to provide the reasoning system with the capability to analyze the image data as well as the textual data to determine an answer to extract desired information such as answering a question pertaining to the textual data and image data. Accordingly, existing reasoning systems often ignore these images despite their obvious utility for answering the question.